Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ILM to DSL migration test for BA. #106054

Merged
merged 3 commits into from
Mar 7, 2024

Conversation

afoucret
Copy link
Contributor

@afoucret afoucret commented Mar 7, 2024

This PR is a fix for: #104470

The fix consists of an additional verification on the index template existence (or version) before launching the tests.
This additional verification give enough of time time to cluster to finish initializing (or updating.

@afoucret afoucret added >test Issues or PRs that are addressing/adding tests auto-backport Automatically create backport pull requests when merged :EnterpriseSearch/Application Enterprise Search Team:Enterprise Search Meta label for Enterprise Search team v8.13.1 v8.14.0 v8.12.3 labels Mar 7, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/ent-search-eng (Team:Enterprise Search)

@afoucret
Copy link
Contributor Author

afoucret commented Mar 7, 2024

Tested using:

./gradlew ':x-pack:plugin:ent-search:qa:full-cluster-restart:v8.11.0#bwcTest' -Dtests.class="org.elasticsearch.xpack.application.FullClusterRestartIT" -Dtests.bwc=true 

Copy link
Contributor

@demjened demjened left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, minor comments and questions

assertEquals(true, dataStream.evaluate("lifecycle.enabled"));
assertEquals("180d", dataStream.evaluate("lifecycle.data_retention"));
assertEquals("Data stream lifecycle", dataStream.evaluate("next_generation_managed_by"));
assertEquals(false, dataStream.evaluate("prefer_ilm"));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess these checks are better indicators than ilm_policy being null?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test was not correct and in fact ilm_policy is not null but the policy is not used since prefer_ilm is false.

We was not able to see it before since the test were not executed.

My plan is to open a tech debt issue to check is we can completely remove the ILM policy since it is not used anymore.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the time I believe we had to keep it for BWC reasons?

Copy link
Contributor Author

@afoucret afoucret Mar 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have never been fully convince that we have to keep it.
I mean

  • Clusters created before 8.12 will have the ILM policy created before they update and data streams created before the update will use it.
  • The registry update does not cleanup useless policies

I created atech debt issue to evaluate if we can remove it.

if (minVersion != null) {
String pathToVersion = "index_templates.0.index_template.version";
ObjectPath indexTemplatesResponse = ObjectPath.createFromResponse(response);
assertEquals(minVersion, indexTemplatesResponse.evaluate(pathToVersion));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible that in the future the index template gets upgraded to a version number > 3? If so, check a greater-or-equals-than relation instead.

@afoucret afoucret merged commit 281dab5 into elastic:main Mar 7, 2024
14 checks passed
afoucret added a commit to afoucret/elasticsearch that referenced this pull request Mar 7, 2024
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
8.13
8.12 Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 106054

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Automatically create backport pull requests when merged backport pending :EnterpriseSearch/Application Enterprise Search Team:Enterprise Search Meta label for Enterprise Search team >test Issues or PRs that are addressing/adding tests v8.12.3 v8.13.1 v8.14.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants